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Latent class models for mastery testing differ from 
continuum models in that they do not postulate a latent mrstery 
continuum but conceive mastery and non-mastery as two latent classes, 
each Characterized by different probaoilities of success. Several 
researchers use a simple latent class model that is basically a 
simultaneous application of the binomial error model to both mastery 
classes, w. A. Reulecke (1977) presents a version of this model that 
assumes that non-masters guess blindly, with a probability of success 
equal to the reciprocal of the number of alternatives. Assuming a 
loss ratio, these models enable the derivation of an optimal cutting 
score for separating masters from non-masters. To compute this 
cutting score, the model parameters must be estimated. J. a. Emrick 
and P. N. Adams (1969) suggest a method that is based on the average 
inter-item correlation but which, due to its assumptions, is only of 
restricted applicability. The sample applies to the maximum 
likelihood method in as much as this involves estimation equations 
that can be solved iteratively. In this paper, the method of moments 
is used to obtain "quick and easy" estimates. An endpoint that 
assumes that the parameters can simply be estimated from the tails of 
the sample distribution is discussed. A Monte Carlo experiment 
demonstrates t" at the method of moments yields excellent estimators 
and beats the endpoint method uniformly. Five data tables are 
included. (Author/TJH) 
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Abstract 



Latent class models for mastery testing differ from continuum models in that 
they do not postulate a latent mastery continuum but conceive mastery and 
nonmastery as two latent classes each characterized by different probabili- 
ties of success for the test items. Several authors, including Emrick and 
Adams (1969). Besel (1973). Davis. Hickman, and Novick (1973). and Macready 
and Dayton (1977). give a simple latent class model which is basically a 
simultaneous application of the binomial error model to both nastery clas- 
ses. Reulecke (1977) presents a version of this model in whirh it is 
assumed that the nonmasters guess blindly, with a probability of success 
equal to the reciprocal of the number of alternatives . Assuming a loss 
ratio, these models enable us to derive an optimal cutting score for 
separating masters from nonmasters. In order to compute this cutting score 
the model parameters must be estimated. Emrick and Adams suggest a method 
which is based on the average inter-item correlation but which, because of 
its assumptions, is only of restricted applicability. The sample applies to the 
maximum likelihood method inasmuch this involves estimation equations which 
can only be solved iteratively. In this paper it is shown how the method 
of moments can be used to obtain "quick and easy" estimates. In addition, 
an endpoint is discussed which assumes that the parameters can simply be 
estimated from the tails of the sample distribution. A Monte Carlo experiment 
demonstrated that the method of moments yields excellent estimators and beats 
the endpoint method uniformly. 

Key-words: Mastery Testing. State Models. Random Guessing. Latent Class 
Analysis 
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Simple Estimators for the Simple 
Latent Class Mastery Testing Model 



All existing models in mastery testing can be classified as either 
continuum or state models (Meskauskas, 1976). Models of the former type 
postulate a latent or true score continuum underlying the observed test 
score and assume that a point can be denoted dividing the continuum into a 
mastery and a nonmastery region. State models differ from continuum models 
in that they do not postulate a latent continuum but conceive mastery and 
nonmastery 'is two latent classes eich characterized by different probabili- 
ties of a correct response to the test items. Ideally, these probabilities 
would be equal to one for a master and to zero for a nonmaster, but the 
influence of extraneous factors can be expected to introduce a bias making 
these probabilities differ from their ideal values. 

Besel (1973) and Macready and Dayton (1977) have given a latent class 
model for mastery testing which can be regarded as the most general model 
known so far. For each of the two latent classes, it has a different para- 
meter for each of the items representing the probability of a successful 
reply to the item. Denoting the probability of a successful reply to item 
i for a nonmaster and a master by a^. and 8^, respectively, this model des- 
cribes the probability of the jth response vector on an n-item test as 
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where y represents the probability of a master, and a. . = 1 if the response 
to the ith item for the jth vector is right and a.. = 0 otherwise. The ob- 
served score distribution generated by (1) is a mixture of two compound 
binomial distributions (Walsh, 1953, 1959, 1963; Lord & Novick, 1968, sect. 
23.10) with u as mixing parameter. 

The simple latent class model for mastery testing follows from (1) on 
assuming a^. = a and 8^. = 8 for all items. Written not in response vector 
form but as a model for the observed score distribution, it is equal to 

(2) Ptx) = (;)[(l-li )a^l - a)"-^ + u8'(l - 8)"-']. 

From this expression it is clear that this model involves an observed score 
distribution which is a mixture of two simple binomial distribution with success 
parameters a and 8 and mixing parameter \i. The simple latent class ircdci was 
introduced in the area of mastery testing by Emrick and Adams (1969), and 
also given by Davis, Hickman, and Novick (1973) and Macready and Dayton (1977). 

Dayton and Macready (1976) consider a further simplification of (2) which 
is obtained by assuming a = 1 - This amounts to the notion that the devia- 
tions of a and 8 from their ideal values of one and zero are equal to each 
other. They also show the possibility of incorporating hierarchical relations 
into the aforementioned models and using them for validating behavioral 
hierarchies. 

An important property of the simple latent class model is that it allows 
the derivation of a simple rule for separating masters from nonmasters. This 
rule, which in statistical terminology is a monotone, nonrandomized Bayes rule, 
has the form of a cutting score on the test, c say, so that students with 
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X > c are declared to be a master and those with X < c a nonraaster. This 
cutting score is equal to 



where X is the loss ratio 9,^/1^, being the loss associated with misclassi- 
fying a nonmaster and £j with misclassifyihg a master (Emrick & Adams, 1969^ 
for a derivation, see also van der Linden, 1980a). 



In order to be able to use the above cutting score, the model para- 
meters a, 6, and y must be estimated from a sample of test scores. 
In this paper we will review the use of some of the available estimation 
methods for the simple latent class model and report results from a Monte 
Carlo investigation into the statistical properties of two classes of 
estimators — one known as moment estimators and another as endpoint 
estimators. But before proceeding, we observe that the optimal cutting 
score given in (3) can also be used when not a latent class model but a 
continuum model with indifference zone is adopted. In that case no para- 
meter estimation is involved. This possibility has been worked out else- 
where (van der Linden, 1980b), and will not be considered here. 

Emrick and Adams (1969) and Emrick (1971) show how estimators for a 
and B can be obtained via the square root of the interitem correlation. 
Their method does not yield an estimator for p but, on the contrary, assumes 



(3) 




In 



«(l-g) 
(l-a)e 



Estimating the Model Parameters 
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that y is a priori known and has a value equal to .50. It also assumes an 
a priori known ratio of the two success parameters, a and B. Since there 
will hardly be situations for which these two assumptions hold. Emrick and 
Adams' method is of restricted applicability and will be disregarded further 
in this paper. 

From a statistical point of view, maximum likelihood estimation is an 
attractive method. The use of maximum likelihood estimation for the para- 
meters of the simple latent class model for mastery testing has become 
available through the computer programs by Dayton and Macready (1978; see 
also. Macready & Dayton. 1979). Maximum likelihood estimates for the para- 
meters of this model are by no means simple, however. The estimation 
equations are intractable and can only be solved using iterative procedures 
as implemented in Dayton and Macready's programs. Since in this paper the 
emphasis is on simple estimators, to be computed by hand or with the aid 
of a pocket calculator, the possibility of maximum likelihood estimation 
will, in spite of its favorable properties, be disregarded further in this 
paper as well. 

As noted earlier, the model given in (2) is a mixture of two simple 
binomial distributions. Mixtures of binomials have been extensively studied 
in the statistical literature (e.g.. Blischke. 1962, 1963. 1964; Pearson. 
1915; Rider. 1961). and on several independent occasions the method of moments 
has been used to derive estimators for mixtures of two binomials with results 
applying to the mastery testing model dealt with in this paper. 

Generally, the method of moments expresses the parameter to be estimated as 
explicit functions of population moments and next substitutes the corresponding 
sample moments to obtain estimators for these parameters. The method of 
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moments ordinarily yields simple, closed-form estimators, which are con- 
sistent under mild conditions (Rao. 1972. p. 351). 

As can be verified from, for example. Blischke (1962. 1964). the method 
of moments yields the following estimators for mixtures of two binomials 

(4) a= ^A - 1(a2. 4AFi + 4F2)1/^. 

(5) g = 1a + 1(a2 - 4AF . 4F,)l/2. 



(6) n = 



with 



Fi - a 



I 

a 



(J) r 1 ? - l)---(>« - k + 1) 

k-i/, -O(x); 

•'=k n(n - l)...(n - k + 1) 



(8) A = 



and where m is the sample size. 0(x) the observed frequency of X = x and 
F,^ is an expression which is. up to the factor (n - l)...(n - k + 1) in the 
denominator, equal to the kth factorial moment (see also. Johnson & Kotz. 
1969. sect. 3.11). For data sets normally encountered in mastery testing 
(7) can be computed easily by hand for k = 1. 2. 3. Once A has been calcul- 
ated from (8). (4) - (6) give the desired estimates for a. 8. and u. 



simple Estimators 
7 



In some applications of the model in this paper, it may be meaningful 
to take a "mastery or random guessing" point' of view and to assume that 
the student?! either master the items or not master them and guess blindly. 
In that case the success parameter a can be treated as a known parameter 
and set equal to reciprocal of the number of item alternatives. This point 
of view has been taken by Reulecka (1977a. 1977b) and was alluded to in an 
example by Emrick (1971). If a may be supposed to be a known parameter, the 
method of moments can still be used but now leads to a different set of 
estimators since only the first two sample moments are needed for estimating 
the remaining success parameter. 8, and the mixing parameter u. 

As can be verified from van der Linden (1980c). the method of moments 
yields the following estimators for mixtures of two binomials with one known 
success parameter 



(9) e = 



O 



(^2 - 



(10) li = 



o 

B - a 



where Fj and are again obtained from (7) by substituting k = 1. 2. 

Reulecke (1977a) has proposed a method of estimation in which a is 
treated as a known parameter and 8 and m are estimated from the frequencies 
in the right-hand tail of the sample distribution. This method can also be 
used when a is to be considered an unknown parameter, however. Then a is 
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estimated similarly from the left-hand tail of the sample distribution 
(van der Linden, 1980b). 

More in particular, this "endpoint" method of estimation assumes that 
the tails of the sample distribution are virtually unmixed and that the 
parameters can be estimated from the ratio of the observed frequencies in 
the two outmost categories in each tail as 



(11) 



a 



0(1) 
nO(0) + 0(1) 



(12) 



8 = 



nO(n) 



O(n-l) + nO(n) 



(13) Z = 0(n) 



■O(n-l) + nO(n) 
nO(n) 



m-1 
m , 



where m still denotes the sample size, n the test lenght. and 0(.) the ob- 
served frequences. 

The estimator for u in (13) is based on the observed frequences of 
X = n - 1 and X = n. It is also possible to derive an estimator for u from 
the frequences of X = 0 and X = 1. However, the problem this creates can be 
circumvented by using neither of the two and instead substituting a and y 
in the first moment equation. This results in 



(14) 



y = 



e - a 
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(van der Linden, 1980a). 

We finally observe that each of the above sets o? estimators can be 
ur,ed to obtain an estimate of the cutting score c* given in (3). 

Results from Monte Carlo Experiments 

Using random procedures from the NAG Fortran Library (1977), Monte 
Carlo experiments were run in which we varied the two success parameters, 
the mixing parameter, test length, and sample size and determined the conse- 
quences for the expected error of estimation and the risk function using 
squared error loss. This was done for both sets of moment estimators, the 
endpoint estimators and the corresponding estimators for the optimal cutting 
score c*. 

The results can be found in Tables 1 - 5 at the end of this paper. In 

these tables e is a generic symbol for an error of estimation and Ee and 
2 

Ee denote the estimated expected error of estimation and the estimated 
risk function, respectively. As indicated by their subscripts, the results 
for the estimators of the optimal cutting score c* are reported for loss 
ratio values equal to .25, 1, and 4. Each figure in Tables 1 - 5 is based on 
1,000 replications. 

From the results presented in Tables 1 - 5 the following conclusions 
can be drawn: 

(1) On the whole, the results for the moment estimators with a unknown 
(equations 4-6) are excellent and suggest estimators which can safely be 
used in most practical situations. The results are better the larger the 
difference between a and 8, the closer \i to .50, the longer the test, and 
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the larger the sample size. To see a result typical of what was encountered 
in our experiments, for example, the center column of Table 3 can be used. 
The values of Ee and Ee^ for I and J differ only in their third decimal 
from zero, while the values for C». although somewhat larger, are almost 
equally good. The only exception to this favorable conclusion has to be 
made for the results for I and for (a. 8) = (.40. .60) in Table 1. This 
set of parameter values .epresents a situation in which our model comes close 
to a single binomial, and in that case it is clearly difficult to estimate 
the mixing parameter and the optimal cutting score reliably. 

(2) The results for the moment estimators with a known (equations 
9 - 10) are even better than those with a unknown. Especially the results 
for S» show a considerable improvement in their Ee and Ee^ values compared 
with the results for C*. The same trends as were observed for the case with 
a unknown can be seen in Tables 1-4: The values of Ee and Ee^ tend to zero 
when the difference between a and 8. the test length, and the sample size 
increase and when u goei to .50. Inspecting Table 1. it appears that the 
moment estimators with a known get less upset when a and 8 approach each 
other in value and the situation resembles that of a single binomial. Only 
the values of . the risk function for c» are still too large, indicating that 
in this situation inaccurate estimates of c* can be expected. 

(3) The Monte Carlo results for the endpoint estimators show that 
these are completely unreliable. From the great variety of parameter sets 
we used, three are picked out and given in Table 5. Parameter set I 
(a = .10. 8 = .90. u = .70. n = 10. m = 100) is the set with the best results 
we have seen, parameter set II (a = .25. 8 = .75. u = .70. n = 10. m = 25) 
was one of the worst sets, while results typical of what we normally 
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encountered were obtained for parameter set III (a = .25, 6 ^ .75, y = .50, 

n = 10, tn = 100). Even for parameter set I the results are still too poor for 
practical purposes. 

(4) From Table 5, it can be concluded that the use of the endpoint/ 
moment estimator for y defined in (14) is in all cases a considerable 
improvement on the use of y. As can be seen in the last three rows of this 
table, substituting y instead of y into the optimal cutting score results 
in a remarkable gain in efficiency but not enough to yield an estimator 
with errors of estimation that can be tolerated when used in practice. 

(5) During the Monte Carlo experiments it was counted how often 
inadmissible estimates, i.e., eLc.nates lying outside the interval the 
parameter is defined on, were met. For the moment estimators with a unknown 
only one percent of the replications resulted in such estimates, while 
such estimates were not at all encountered for the moment estimators with 

a known. The situation was completely different for the endpoint estimators 
y and y, however. Percentages of inadmissible estimates exceeding 20 or 30 
per cent were no exception, and in one case no less than 47 per cent of 
the replications yielded values for y larger than one. 

(6) A different problem was met for the endpoint estimators a and 3. 

As is clear from (11) and (12), these expressions are indeterminate whenever 
the tails of the sample distribution are empty. This happened very often and 
especially for a and g values close to each other, y values far away from 
.50, small samples, and long tests. The largest percentage of incetermin-^te 
estimators found in one experiment was equal to 48 per cent. 
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Conclusion from the Results 



The foregoing suggests that the moment estimators can safely be used 
in most practical situations. The only exception which has to be made is 
for the case parameter values can be expected for which the model comes 
close to a single binomial. In that case less accurate estimates of the 
mixing parameter and the optimal cutting score can be expected. When a can 
be set equal to the reciprocal of the number of response alternatives, even 
better results can be expected than when a is unknown, especially when the 
model comes close to a single binomial. It must be taken into account, 
however, that this only applies to the extent the model of blind guessing 
holds for the nonmasters. If this is not the case, then the value of a is 
specified incorrectly and the moment estimators given in (9) - (10) may, 
in fact, be worse than those for a unknown. 

It is not recommended to use the endpoint estimators in any situation. 
Not only were they beaten by the moment estimators for all parameter sets 
we used, they are also likely to It^ad to situations in which the estimates 
are indeterminate or, if not so, take inadmissible values. The endpoint/ 
moment estimator for \x given in (14) offers an improvement, but is still 
too poor for applications in practice. 
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TABLE 1 

Results for Moment Estimators with Varying Success 
Parameters and y = .70, n = 10, and m = 100 
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TABLE 2 

Results for Moment Estimators with Varying Mixing 
Parameter and a = .25, B = .75, n = 10, and m = 100 
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TABLE 3 

Results for Moment Estimators with Varying Test 
Length and a = .25, g = .75, y = .70, and m = 100 
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.004 
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.182 
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TABLE 4 

Results for Moment Estimators with Varying Sample 
Size and a = .25. 8 = .75. y = .70. and n = 10 
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.007 
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moment estimator with a unknown 
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TABLE 5 

Some Results for the Endpoiat Estimators 
Parameter Set I II m 





Ee 


Ee2 


Ee 


Ee2 


Ee 


Ee2 


a 


.009 


.002 


.386 


.346 


.072 


.053 
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-.004 


.001 


-.240 


.219 


-.062 


.041 




.025 


.015 


-.180 


.140 


.102 
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• 
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.119 


.015 
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C* 

^.25 
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.842 
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-1.507 
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c* 
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c* 
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6.585 


c* 


.011 
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-.168 
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-.009 
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c* 


.004 


.241 
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1.582 


.026 
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~ endpoint estimator 

• endpoint/moment estimator 



ot = .10. 8 = .90, p = .70. n = 10. m = 100 
ot = .25. 8 = .75, M = .70. n = 10. m = 100 
a T .25. 8 = .75. p = .50. n = 10. m = 100 
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